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Be it known that we, Don R. Auten, a citizen of the United States, 
residing at 8101 Stewarts Ferry Parkway, Nashville, TN 37214; Richard T. 
Akers, a citizen of the United States, residing at 4841 Cimarron Way Drive, 
Antioch, TN 37013; and Richard Gembar, a citizen of the United States, 
residing at 1102 Woodridge Place, Mt. Juliet TN 37122; have invented a new 
and useful "System and Method for Generating and ControUing a Simulated 
Musical Concert Experience." 



The present invention relates generally to audio and video simulations 
of a pre-recorded musical performance. 

More particularly, this invention pertains to the generation and control 
of a simulated musical concert experience and participation by a musician in a 
pre-recorded musical performance using a musical instrument as the control 
device. 

Virtual reality systems are generally recognized to be a combination of 
computer hardware, software, and peripherals which recreate a virtual world 
or virtual environment using a video display, often in combination with an 
audio sound system. Conventional virtual reahty systems are quite complex, 
combining the hardware, software, and peripheral devices in a specific manner 
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to immerse or subject the user of the system to visual and sound stimuh which 
simulate a real world experience. Typically, a virtual reaUty system further 
includes one or more input devices and interface software so that the user of 
the system can interact with the virtual environment that is being recreated, 
such as to simulate the user movement in the environment or manipulation of 
virtual objects reproduced in the virtual environment. 

Virtual reahty systems in the prior art have been used for 
entertainment purposes, to conduct scientific experiments, or to allow a user to 
indirectly carry out tasks which would otherwise be too difficult or dangerous 
when conducted in a real environment. 

To a lesser extent, virtual reahty systems have been used to create and 
control a virtual world that responds to music signals or to pre-recorded control 
tracks corresponding to music signals. Such a system is disclosed in U.S. 
Patent No. 5,513,129, which describes a virtual reality in which a music source 
is connected to an electronic interface and to a virtual reahty processor. The 
system is further controlled by one or more input devices, such as a head 
tracker and manipulator glove. The pre-recorded music, along with an optional 
pre-recorded control track, controls and manipulates objects within the virtual 
environment such that the music effectively drives the display of an animated 
graphical scene. However, the system described in U.S. Patent No. 5,513,129 
does not provide a simple and effective method for allowing a musician to 
participate in and control a "virtual environment" through the actual operation 




of a musical instrument, such as an electric guitar. Such a system would 
provide a source of entertainment to professional and amateur musicians alike. 
It also would assist musical instrument manufacturers in promoting the sale of 
their instruments by allowing a prospective purchaser to recreate a musical 

5 concert, to simulate the musician's participation in the concert, and to control 
the sound portion of the concert through operation of the guitar or other 
instrument. Preferably, such a simulation system would minimize the use of 
complex and expensive hardware and software so that the system would be 
easy to set up and affordable even at the retail store level. Such a system is 

10 lacking in the prior art. 



SUMMARY OF THE INVENTION 

In the simulation and control system of the present invention, the video 
and sound portions of a musical performance or concert is pre-recorded on a 

15 video tape, digital disc, or other media containing audio and video tracks. The 
sound portion of the concert will include a separate instrument track 
representing musical sounds that would be made during the pre-recorded 
concert by a specific musical instrument. The tape or disc containing the pre- 
recorded video and sound tracks is loaded into a conventional video disc or 

20 video tape player. The video output from the video player is connected to a 
video display, such as a stereoscopic headset. The audio output of the video 
player is connected to left and right audio inputs on a multi-channel audio 
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mixer. Means are provided in the mixer or in a decoder to separate the pre- 
recorded instrument sound track from the left and right concert sound tracks 
containing music from the other concert instruments as well as ambient crowd 
and backstage noise. The separated instrument sound track is then available 
for control by other system hardware. 

A musical instrument corresponding to the specific musical instrument 
represented by the pre-recorded instrument sound track has its audio output 
connected to an instrument input on a system interface box. The interface box 
also includes an instrument track audio input connected to an output on the 
mixer, with a controlled instrument track audio output fi-om the interface box 
connected to a separate audio input on the mixer. 

The interface box includes an instrument track control circuit which 
electronically varies an electrical characteristic or parameter of the instrument 
track audio, such as the audio level. The instrument track control circuit in 
the interface box is responsive to the instrument audio signals received at the 
instrument audio input on the interface box. Accordingly, the playing of the 
musical instrument by the musician will control or vary the sound level of the 
instrument track provided to the mixer. The mixer combines the controlled 
instrument soundtrack with the left and right concert sound tracks, and 
provides the mixed audio to a mixer output connected to the speakers on the 
headset. The musician can then cause the system to vary the volume level of 
the instrument track in response to playing of the musical instrument. 



In a further embodiment of the system, the interface box includes a 
bypass circuit controlled by an external switch. For a normal system mode, 
only the controlled instrument sound track is provided to the mixer and 
therefore to the headset. In a bypass mode, as selected by the switch, the 

5 bypass circuit causes the interface box to suppress the instrument sound track 
and to provide the audio signals produced by the instrument directly to the 
mixer. In this bypass mode, then, the musician can hear himself play the 
instrument in synchronization with the concert video track and the left and 
right concert sound tracks, thereby enhancing the level of simulated 

10 participation. Preferably, the video portion of the pre-recorded concert is 
filmed as if "through the eyes" of an onstage musician so that the user of the 
system can assume that role while playing the instrument. 



BRIEF DESCRIPTION OF THE DRAWINGS 

15 ^ Fig. 1 is a block diagram of the simulation control system of the present 

invention. 

\^ Fig. 2 is a schematic diagram of the system interface box used in the 
system of Fig. 1. 



20 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The concert simulation and control system 10 is shown generally in Fig. 
1. A musical instrument, such as a guitar 20, having one or more pick-ups or 

-5- 



other transducers that will generate electrical audio signals, when the guitar is 
played, at an instrument audio output 21. Instrument audio output 21 is 
electrically connected to an instrument input 22 on a system interface box 18. 
Interface box 18 includes an instrument audio output 23 which, as will be 
described below, can be switched to provide a direct electrical connection to 
instrument input 22. 

In one embodiment of system 10, where the musical instrument is guitar 
20, a musical effects processor 17 is electrically connected between instrument 
audio output 23 on interface box 18 and a corresponding mixer instrument 
input 26 on a multi-channel audio mixer 16. Effects processor 17 is of 
conventional design and will typically include electronically induced distortion, 
delay, and other special effects which electrically modify the audio signals 
generated by guitar 20. 

The audio mixer 16 is also of conventional design and in one 
embodiment, will have eight audio channels. A pair of left and right source 
audio inputs 31 on mixer 16 are electrically connected to corresponding left and 
right source audio outputs 15 on an audio video ("AV") playback device 14. AV 
playback device 14 is also of conventional design, and can be a video tape 
player or DVD player. Accordingly, AV player 14 will also have a source video 
output 13 electrically connected to a video input 32 on a video display device, 
such as the video stereoscopic headset 11. Preferably, headset 11 will be a 
conventional head mounted display wearable by the player of guitar 20. 



Headset 11 will include left and right speakers driven by a pair of left and 
right headset audio inputs 33 which are electrically connected to corresponding 
left and right mixer audio outputs 12 on mixer 16. Three-dimensional viewing 
of the concert video is enabled by conventional 3D shutter glasses (not shown) 
5 inside the headset. A commercially available headset usable in this 
appHcation is the Model CE200-W Cyber Eye headset from General Reality 
Company, San Jose, CaUfornia. 

The audio portion of the pre-recorded musical concert to be played back 
by AV player 14 will include left and right concert sound tracks and a 

10 separable instrument sound track, all of which are electrically transmitted to 
mixer 16. Mixer 16 can include conventional circuits capable of electronically 
separating the instrument sound track from the left and right concert sound 
tracks. Otherwise, the tracks are separated externally by AV player 14 or by 
an external decoder and provided to mixer 16 through discrete inputs. 

15 Accordingly, mixer 16 will further include an instrument track mixer output 27 
electrically connected to an instrument track interface input 25 on interface 
box 18. In accordance with the novel control system and method of this 
invention, and as will be described further below, interface box 18 will 
preferably include a controlled instrument track audio output 28 electrically 

20 connected to input 29 on mixer 16. Mixer 16 can then, in conventional fashion 
well known to those of skill in the art, combine and mix the left and right 
concert sound tracks provided at input 31 by AV player 14 with the controlled 
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instrument sound track provided at input 29, and output the mixed audio 
signal to the headset 11 at mixer audio output 12. 

Interface box 18 can also include a switch input 30 electrically connected 
to an external switch 19 operable by the musician playing guitar 20. In 
5 conjunction with the b5^ass and control circuits of system interface box 18 as 
illustrated in Fig. 2 and described below, switch 19 will place interface box 18 
in either a normal or bypass mode, allowing the musician to select from at 
least two different simulation modes available on system 10. 

One embodiment of the system interface box 18 is shown schematically 

10 in Fig. 2. The instrument input 22 is electrically connected to a relay RLl. 
Relay RLl has a "normal" position in which it will cause the instrument audio 
output 21 to be electrically connected to the inverting input (pin 2) of operation 
amplifier ICl, through resistor Rl. Relay RLl can be switched to a "bypass" 
mode whereby the instrument audio output 21 is electrically connected directly 

15 to instrument audio output 23 on interface box 18. Relay RLl is caused to 
switch from the normal mode to the bypass mode by activation of switch 19 
connected to interface box 18 at switch input 30. Preferably, switch 19 is a foot 
operated switch so that the musician can change the system mode while 
simultaneously playing guitar 20 with both hands. 

20 When the system 10 and bypass circuit 18 are in the normal mode, the 

instrument audio signals are amplified by ICl in a conventional manner, with 
the gain set by resistors R3 and Rl. The audio signals at the output of 



amplifier ICl are rectified by diodes Dl and D2. The ripple in the output 
signal fi:om amplifier ICl is reduced by capacitor Cl, so that the output is a 
substantially DC voltage having a magnitude that corresponds to variations in 
the average peak magnitude of the audio signals fi:om guitar 20. The rectified 
signal is applied to one input of an analog comparator IC2. The output of 
comparator IC2 is either "high" or "low" depending upon the relationship 
between the voltage at terminal 3 of IC2 and the voltage at terminal 2. The 
voltage at terminal 2 of IC2 is a comparator threshold set in conventional 
manner by potentiometer VRl connected to a nominal five volt supply voltage. 
The output of comparator IC2 controls an electronic switch IC3, which can be a 
conventional CMOS switch, such as a type CD4066 available from National 
Semiconductor. 

Electronic switch ICS performs the function of switching instrument 
track input 25 on interface box 18 into the control circuit of interface box 18 
such that the instrument sound track audio is electrically connected to 
controlled instrument track output 28 through a control device LDRl. Control 
device LDRl can be any conventional signal conditioning device, such as an 
amplifier or an active or passive attenuator. In the embodiment of Fig. 2, 
control device LDRl is a hght dependent resistor, such as the type CLM5500 
available from Javenco Electronics Supply. Accordingly, the instrument sound 
track audio signals are attenuated by a resistive portion of control device 
LDRl, which is varied by the output from a light emitting portion. Thus, 
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control device LDRl includes a light emitting diode having a control input 
driven by the output of amplifier ICl. Again, the output of amplifier ICl is 
rectified by diodes D4 and D5, with the rectified signal further smoothed by 
capacitor C2, such that the variable DC voltage is appUed across potentiometer 
VR2, Accordingly, as the RMS or average signal level or magnitude of the 
instrument audio signal increases, the voltage across potentiometer VR2 will 
proportionally increase, thereby increasing the current through the LED 
portion of control device LDRl. This decreases the attenuation provided by 
control device LDRl of the instrument sound track audio between instrument 
sound track audio input 25 and controlled instrument sound track audio 
output 28. Using such a control circuit, then, and when the system 10 is in the 
normal mode, the musician will hear the pre-recorded instrument sound track 
on the headset at a volume that varies in proportion to how hard or how soft 
the musician is striking the strings of guitar 20. This control effect provides a 
simulation of the musician actually participating as a player of the instrument 
in the pre-recorded musical concert. 

In a further embodiment of system 10, an additional switch output can 
be provided on switch IC3 and connected to a control input on AV player 14, 
such that generation of instrument audio signals by playing of the guitar 20 
will activate playback of the pre-recorded musical concert from AV player 14. 

In yet another embodiment of the system 10 in which AV player 14 is a 
DVD player, having a conventional PCM/AC-3 digital audio output at source 
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audio output 15, a Dolby Digital Decoder can be placed in the audio circuit 
between source audio output 15 and mixer source audio input 31, to digitally 
separate the pre-recorded instrument sound track from the pre-recorded left 
and right concert sound tracks. 

Although system 10 is shown and described for use with an electric or 
ampHfied acoustic guitar, it can be used with a variety of other musical 
instruments which either directly, or indirectly through an interface device, 
will produce electrical audio signals representative of the sounds made by the 
instrument. 

Also, in the embodiment of system 10 as described and shown, the 
characteristic of the pre-recorded instrument sound track that is controlled by 
the control circuit in interface box 18 is the signal level or magnitude of the 
audio. However, by changing the nature of the control circuit, different 
parameters of the instrument sound track audio can be varied in response to 
operation of the musical instrument. For example, a controllable analog or 
digital audio filter could be substituted for the light dependent resistor shown 
in Fig. 2, whereby the frequency response of the filter will vary in accordance 
with the voltage across potentiometer VR2, thereby changing the frequency or 
tonal characteristics of the instrument audio track as the guitar 20 is played. 

In many applications, a wearable stereoscopic headset with left and 
right speakers is an ideal combination of a video display and audio transducer 
to be used with system 10. However, in other applications, a two dimensional 
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video display can be used, either wearable or not, in conjunction with floor or 
wall mounted audio speakers. Or, multiple audio and video playback and 
display systems can be used in parallel. 

Generally, the pre-recorded musical concert will be recorded on video 
using a stereoscopic camera to produce a "3D" playback effect, with 
simultaneous audio recording of the concert sound tracks and, usually, the 
instrument sound track. Other backstage footage can be included to simulate 
the musician's participation in pre-concert preparation and build-up. To 
further create the virtual concert experience, additional backstage and onstage 
audio can be recorded, either during filming of the actual musical performance 
or later in the studio. The separate instrument sound track can be recorded 
live in conjunction with the video and other audio portions of the musical 
concert or can be added later or re-mixed in the studio. If the system 10 is to 
be used by an instrument manufacturer to promote the sale of its products, 
suitable marketing and promotional logos and messages can be superimposed 
over the concert video and/or audio while the system is in operation. To this 
end, the pre-recorded video can include a segment where the musician is shown 
selecting a specific manufacturer's instrument to play on stage. 

One advantage of this system is that no computer is needed to operate or 
control it. If a DVD player is used for AV player 14, multiple pre-recorded 
concert segments can be placed on the disc, allowing the user of the system to 
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easily switch to other programs (a jazz club, a country music festival, etc.), 
representing a favorite experience, venue or band. 

Thus, although there have been described particular embodiments of the 
present invention of a new System and Method for Generating and Controlling 
a Simulated Musical Concert Experience, it is not intended that such 
references be construed as Hmitations upon the scope of this invention except 
as set forth in the following claims. 
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